The traditional reinforcement learning algorithm has limitations in convergence speed and solution accuracy when solving the taxi path planning problem and the traffic signal control problem in traffic scenarios. Therefore， an improved reinforcement learning algorithm was proposed to solve this kind of problems. Firstly， by applying the optimized Bellman equation and Speedy Q-Learning （SQL） mechanism， and introducing experience pool technology and direct strategy， an improved reinforcement learning algorithm， namely Generalized Speedy Q-Learning with Direct Strategy and Experience Pool （GSQL-DSEP）， was proposed. Then， GSQL-DSEP algorithm was applied to optimize the path length in the taxi path planning decision problem and the total waiting time of vehicles in the traffic signal control problem. The error of GSQL-DSEP algorithm was reduced at least 18.7% than those of the algorithms such as Q-learning， SQL， Generalized Speedy Q-Learning （GSQL） and Dyna-Q， the decision path length determined by GSQL-DSEP algorithm was reduced at least 17.4% than those determined by the compared algorithms， and the total waiting time of vehicles determined by GSQL-DSEP algorithm was reduced at most 51.5% than those determined by compared algorithms for the traffic signal control problem. Experimental results show that， GSQL-DSEP algorithm has advantages in solving traffic scenario problems over the compared algorithms.